{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 基础理论部分"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 0. Can you come up out 3 sceneraies which use AI methods?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Ans：广告推送、自动驾驶、商品推荐、机器人自动化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. How do we use Github; Why do we use Jupyter and Pycharm;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Ans:Github用来托管代码，可以完成多人合作开发的任务，个人可以通过fork获取他人的代码与文件、通过commit提交对自己与他人的代码修改\n",
    "#### 使用Jupter可以方便使用各种python工具，即时显示函数图像、统计图表等，模拟实验环境\n",
    "#### 使用Pycharm来进行中大型python项目的开发"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. What's the Probability Model?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Ans：概率模型是用来描述不同随机变量之间关系的数学模型，通常情况下刻画了一个或多个随机变量之间的相互非确定性的概率关系。从数学上讲，该模型通常被表达为(Y,P)，其中 Y 是观测集合用来描述可能的观测结果， P 是 Y 对应的概率分布函数集合。若使用概率模型，一般而言需假设存在一个确定的分布P 生成观测数据 Y 。因此通常使用统计推断的办法确定集合 P 中谁是数据产生的原因。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Can you came up with some sceneraies at which we could use Probability Model?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Ans：股票分析、赌博、娃娃机等"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Why do we use probability and what's the difficult points for programming based on parsing and pattern match?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Ans:使用概率模型可以尽可能模拟现实生活的场景；现实事务的数学抽象过程"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. What's the Language Model;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Ans:语言模型是通过一个语音识别器来定义一系列的词的东西。一些语言模型还会给每一个词序列一个权重或者概率，来当做这个词序列的language score，来彰显词序列的重要程度。有了语言模型，不符合语法规则或者看着就不像句子的输出就可以直接被排除掉，识别错误的情况也会大大减小。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Can you came up with some sceneraies at which we could use Language Model?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Ans:Siri等聊天机器人的语言生成"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7. What's the 1-gram language model？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Ans:1-gram假设一个语句中的每个单词是独立的不受其他单词存在的影响"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 8. What's the disadvantages and advantages of 1-gram language model？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Ans:缺点是不符合现实规律；优点是统计简单"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 9. What't the 2-gram models？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Ans:2-gram概率模型是指在一个语句中一个单词的合理性概率受其相邻一个单词的影响，计算合理性概率时要把相邻单词存在的概率计算在内"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 编程实践部分"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 0.设计自己的句子生成器"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 在火影忍者中一个‘忍者’的语言可以定义为："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {},
   "outputs": [],
   "source": [
    "first_grammar = '''\n",
    "ninja => 名词词组 动词词组\n",
    "名词词组 => 量词 形容词* 名词\n",
    "形容词* => null | 形容词 形容词*\n",
    "动词词组 => 动词 名词词组\n",
    "量词 => 一个 | 这个 | 一个村的 |  一个小队的 | null\n",
    "名词 => 螺旋丸 | 螺旋手里剑 | 千鸟 | 无限月读 | 须佐能乎 | 神罗天征 | 鸣人 | 佐助 | 辉夜 | 宇智波班 | 佩恩\n",
    "动词 => 使用 | 击败了 | 打倒了 | 打败了 | 击溃了\n",
    "形容词 => 火之国的 | 水之国的 | 忍术强的 | 幻术强的 | 体术强的 | null\n",
    "'''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_grammar(grammar_str, split='=>', line_split='\\n'):\n",
    "    grammar = {}\n",
    "    for line in grammar_str.split(line_split):\n",
    "        if not line.strip(): continue\n",
    "        exp, stmt = line.split(split)\n",
    "        grammar[exp.strip()] = [s.split() for s in stmt.split('|')]\n",
    "    return grammar"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ninja': [['名词词组', '动词词组']],\n",
       " '名词词组': [['量词', '形容词*', '名词']],\n",
       " '形容词*': [['null'], ['形容词', '形容词*']],\n",
       " '动词词组': [['动词', '名词词组']],\n",
       " '量词': [['一个'], ['这个'], ['一个村的'], ['一个小队的'], ['null']],\n",
       " '名词': [['螺旋丸'],\n",
       "  ['螺旋手里剑'],\n",
       "  ['千鸟'],\n",
       "  ['无限月读'],\n",
       "  ['须佐能乎'],\n",
       "  ['神罗天征'],\n",
       "  ['鸣人'],\n",
       "  ['佐助'],\n",
       "  ['辉夜'],\n",
       "  ['宇智波班'],\n",
       "  ['佩恩']],\n",
       " '动词': [['使用'], ['击败了'], ['打倒了'], ['打败了'], ['击溃了']],\n",
       " '形容词': [['火之国的'], ['水之国的'], ['忍术强的'], ['幻术强的'], ['体术强的'], ['null']]}"
      ]
     },
     "execution_count": 187,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "create_grammar(first_grammar)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "choice = random.choice\n",
    "\n",
    "def generate(gram, target):\n",
    "    if target not in gram: return target # means target is a terminal expression\n",
    "    \n",
    "    expaned = [generate(gram, t) for t in choice(gram[target])]\n",
    "    return ''.join([e if e != '/n' else '\\n' for e in expaned if e != 'null'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'佩恩打倒了一个村的千鸟'"
      ]
     },
     "execution_count": 189,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generate(create_grammar(first_grammar),'ninja')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 在海贼王中一个海贼可以这么形容："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {},
   "outputs": [],
   "source": [
    "second_grammar = '''\n",
    "poacher => 主 谓 宾\n",
    "主 => 路飞 | 索隆 | 香吉士\n",
    "谓 => 吃过 | 拥有\n",
    "宾* => null | 宾 宾*\n",
    "宾 => 橡胶果实 | 面子果实 | 响雷果实 | 霸王色霸气 | 武装色霸气 | 见闻色霸气\n",
    "'''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'香吉士拥有霸王色霸气'"
      ]
     },
     "execution_count": 191,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generate(create_grammar(second_grammar),'poacher')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_n(gram, target,n):\n",
    "    for i in range(n):\n",
    "        print('{}.:{}'.format(i+1,generate(gram,target)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.:香吉士拥有橡胶果实\n",
      "2.:香吉士拥有霸王色霸气\n",
      "3.:索隆拥有见闻色霸气\n",
      "4.:索隆拥有橡胶果实\n",
      "5.:索隆拥有响雷果实\n",
      "6.:路飞拥有霸王色霸气\n",
      "7.:路飞吃过见闻色霸气\n",
      "8.:香吉士吃过响雷果实\n",
      "9.:香吉士吃过见闻色霸气\n",
      "10.:香吉士拥有响雷果实\n"
     ]
    }
   ],
   "source": [
    "generate_n(create_grammar(second_grammar),'poacher',10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.:一个村的幻术强的幻术强的忍术强的佩恩打倒了一个无限月读\n",
      "2.:一个小队的螺旋手里剑击溃了千鸟\n",
      "3.:一个小队的幻术强的螺旋手里剑击败了一个小队的佩恩\n",
      "4.:一个螺旋丸击败了幻术强的水之国的佐助\n",
      "5.:佐助打败了一个水之国的鸣人\n",
      "6.:一个体术强的须佐能乎使用螺旋手里剑\n",
      "7.:一个小队的千鸟打倒了一个辉夜\n",
      "8.:一个体术强的须佐能乎打败了一个小队的水之国的忍术强的忍术强的无限月读\n",
      "9.:一个幻术强的忍术强的幻术强的螺旋手里剑击溃了一个小队的无限月读\n",
      "10.:一个小队的幻术强的忍术强的宇智波班打败了一个小队的水之国的螺旋丸\n"
     ]
    }
   ],
   "source": [
    "generate_n(create_grammar(first_grammar),'ninja',10 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.使用新数据源完成语言模型的训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "metadata": {},
   "outputs": [],
   "source": [
    "filename1 = 'C:/Users/Pinkman/Desktop/AI_Nlp/Class_1_6.30/train.txt'\n",
    "store_filename1 = 'C:/Users/Pinkman/Desktop/AI_Nlp/Class_1_6.30/train_clean.txt'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0 ++$++ disability-insurance ++$++ 法律要求残疾保险吗？ ++$++ Is  Disability  Insurance  Required  By  Law?\\n'"
      ]
     },
     "execution_count": 196,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f = open(filename1,'r',encoding = 'UTF-8')\n",
    "f.readline()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {},
   "outputs": [],
   "source": [
    "def string_clean(string):\n",
    "    '''用于中文语句的筛选'''\n",
    "    return string.strip().split('++$++')[2].strip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "metadata": {},
   "outputs": [],
   "source": [
    "def read_txt(filename,store_file):\n",
    "    '''用于txt文件的读取和文本摘取并保存'''    \n",
    "    f = open(filename,'r',encoding = 'UTF-8')\n",
    "    store_f = open(store_file,'w',encoding = 'UTF-8')\n",
    "    \n",
    "    for line in f.readlines():\n",
    "        data = string_clean(line)\n",
    "        store_f.write(data+'\\n')\n",
    "        \n",
    "    f.close()\n",
    "    store_f.close()\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {},
   "outputs": [],
   "source": [
    "read_txt(filename1,store_filename1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "metadata": {},
   "outputs": [],
   "source": [
    "f1 = open(store_filename1,'r',encoding='UTF-8')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 201,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'法律要求残疾保险吗？\\n'"
      ]
     },
     "execution_count": 201,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f1.readline()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "metadata": {},
   "outputs": [],
   "source": [
    "f1.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 203,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "from collections import Counter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 204,
   "metadata": {},
   "outputs": [],
   "source": [
    "def token(string):\n",
    "    '''regular expression'''\n",
    "    return re.findall('\\w+',string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {},
   "outputs": [],
   "source": [
    "import jieba"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 289,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'法律要求残疾保险吗？债权人可以在死后人寿保险吗？旅行者保险有租赁保险吗？我可以开一辆没有保险的新车吗？人寿保险的现金转出价值是否应纳税？如何报告年金收入？AAA家庭保险涵盖什么？什么是简单的退休计划？'"
      ]
     },
     "execution_count": 289,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f1 = open(store_filename1,'r',encoding='UTF-8')\n",
    "article = f1.read()\n",
    "article_clean = ''.join(str(x) for x in article.split('\\n'))\n",
    "article_clean[:100]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "list"
      ]
     },
     "execution_count": 207,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = token(article_clean)\n",
    "type(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 290,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['法律要求残疾保险吗',\n",
       " '债权人可以在死后人寿保险吗',\n",
       " '旅行者保险有租赁保险吗',\n",
       " '我可以开一辆没有保险的新车吗',\n",
       " '人寿保险的现金转出价值是否应纳税',\n",
       " '如何报告年金收入',\n",
       " 'AAA家庭保险涵盖什么',\n",
       " '什么是简单的退休计划',\n",
       " '社会保险残疾保险是什么',\n",
       " '汽车保险是否预付']"
      ]
     },
     "execution_count": 290,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 209,
   "metadata": {},
   "outputs": [],
   "source": [
    "TOKEN = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "metadata": {},
   "outputs": [],
   "source": [
    "for sencente in a:\n",
    "    TOKEN += jieba.cut(sencente)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 291,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['法律', '要求', '残疾', '保险', '吗', '债权人', '可以', '在', '死', '后']"
      ]
     },
     "execution_count": 291,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "TOKEN[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 292,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('保险', 5013),\n",
       " ('的', 3220),\n",
       " ('人寿保险', 2962),\n",
       " ('什么', 2677),\n",
       " ('吗', 2479),\n",
       " ('是', 2344),\n",
       " ('我', 2053),\n",
       " ('是否', 1862),\n",
       " ('可以', 1704),\n",
       " ('健康', 1513),\n",
       " ('如何', 1294),\n",
       " ('医疗保险', 1269),\n",
       " ('多少', 1252),\n",
       " ('汽车保险', 1189),\n",
       " ('在', 913),\n",
       " ('覆盖', 848),\n",
       " ('你', 827),\n",
       " ('有', 775),\n",
       " ('残疾', 724),\n",
       " ('房主', 714)]"
      ]
     },
     "execution_count": 292,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Counter(TOKEN).most_common(20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {},
   "outputs": [],
   "source": [
    "from functools import reduce"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 214,
   "metadata": {},
   "outputs": [],
   "source": [
    "from operator import add,mul"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {},
   "outputs": [],
   "source": [
    "words_count = Counter(TOKEN)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 293,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('保险', 5013),\n",
       " ('的', 3220),\n",
       " ('人寿保险', 2962),\n",
       " ('什么', 2677),\n",
       " ('吗', 2479),\n",
       " ('是', 2344),\n",
       " ('我', 2053),\n",
       " ('是否', 1862),\n",
       " ('可以', 1704),\n",
       " ('健康', 1513),\n",
       " ('如何', 1294),\n",
       " ('医疗保险', 1269),\n",
       " ('多少', 1252),\n",
       " ('汽车保险', 1189),\n",
       " ('在', 913),\n",
       " ('覆盖', 848),\n",
       " ('你', 827),\n",
       " ('有', 775),\n",
       " ('残疾', 724),\n",
       " ('房主', 714)]"
      ]
     },
     "execution_count": 293,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "words_count.most_common(20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 高频单词的频率分布图像"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "metadata": {},
   "outputs": [],
   "source": [
    "frequenices = [f for w,f in words_count.most_common(100)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 218,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = [i for i in range(100)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 220,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 221,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x22fa1252e80>]"
      ]
     },
     "execution_count": 221,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAH49JREFUeJzt3Xl0nFed5vHvrzZJJdmWZEmOFzlyvGQhmcSJ4hgSQkjISg9ODwTC0njonEkzZKaBZk536D5nOMBwBubMQGAaAoGkcTgMgQ5hYkImwdkwgSy2syd2YnmXl0i2ZMvaVVW/+aNeOWW7tNiWVNJbz+ecOlXvrVtV9/XrU4/uvW+919wdEREpPpFCN0BERApDASAiUqQUACIiRUoBICJSpBQAIiJFSgEgIlKkFAAiIkVKASAiUqQUACIiRSpW6AYMp6amxhsaGgrdDBGRKWXDhg373b12pHqTOgAaGhpYv359oZshIjKlmNmO0dTTEJCISJFSAIiIFCkFgIhIkVIAiIgUKQWAiEiRGlUAmNl2M3vVzF4ys/VBWbWZrTGzzcF9VVBuZvY9M2sys1fM7MKc91kZ1N9sZivHZ5dERGQ0TqQH8H53v8DdG4Pt24HH3X0x8HiwDXA9sDi43QrcCdnAAL4CXAIsA74yGBoiIjLxTmUIaAWwKni8Crgxp/xez3oWqDSz2cC1wBp3b3P3dmANcN0pfP6Q9hzs4du/f5OtrZ3j8fYiIqEw2gBw4PdmtsHMbg3KZrn7XoDgvi4onwvsynltc1A2VPmYO9DZz/eeaGJLa9d4vL2ISCiM9pfAl7r7HjOrA9aY2aZh6lqeMh+m/OgXZwPmVoD58+ePsnlHK0tEAejuT53U60VEisGoegDuvie4bwF+Q3YM/+1gaIfgviWo3gzU57x8HrBnmPJjP+sud29098ba2hEvZZHXYAD09KdP6vUiIsVgxAAws3Izmzb4GLgGeA1YDQyeybMSeDB4vBr4dHA20HLgUDBE9ChwjZlVBZO/1wRlYy4ZH+wBKABERIYymiGgWcBvzGyw/v9x90fMbB3wKzO7BdgJ3BTUfxi4AWgCuoHPALh7m5l9HVgX1Puau7eN2Z7kONIDGFAAiIgMZcQAcPetwPl5yg8AV+Upd+C2Id7rHuCeE2/miSmJRYiY5gBERIYTyl8CmxnJRExDQCIiwwhlAEB2GEiTwCIiQwttACQTUfUARESGEdoAKItHNQksIjKM0AZAUkNAIiLDCnEAxHQWkIjIMEIbAGWaAxARGVZoAyCZ0ByAiMhwQhsAZXH1AEREhhPeANAksIjIsEIbANnfAaTIXplCRESOFeIAiJFx6EtlCt0UEZFJKbQBUBbXmgAiIsMJbQAkB1cF05lAIiJ5hTYA3lkVTD8GExHJJ7QBkExklzro6dccgIhIPiEOAC0MLyIynNAGQJnmAEREhhXaAEgmdBaQiMhwQhsAg6eB6nIQIiL5hTcAdBaQiMiwQhsAg2cBqQcgIpJfaANAQ0AiIsMLbQBEI0ZJLKI1AUREhhDaAIB3rggqIiLHC3kAxDQEJCIyhFAHQFkiSq+GgERE8gp1ACS1MLyIyJBCHQBaF1hEZGihDoCk1gUWERlSqAOgTGcBiYgMadQBYGZRM3vRzB4KtheY2XNmttnMfmlmiaC8JNhuCp5vyHmPLwflb5rZtWO9M8cqi8fUAxARGcKJ9AA+D2zM2f4W8B13Xwy0A7cE5bcA7e6+CPhOUA8zOwe4GXgXcB3wAzOLnlrzh5dMRHU5aBGRIYwqAMxsHvBB4CfBtgFXAvcHVVYBNwaPVwTbBM9fFdRfAdzn7n3uvg1oApaNxU4MRWcBiYgMbbQ9gDuAvwcG11ecCRx098EB9mZgbvB4LrALIHj+UFD/SHme14yLskSU/lSGdMbH82NERKakEQPAzP4CaHH3DbnFear6CM8N95rcz7vVzNab2frW1taRmjcsLQspIjK00fQALgU+ZGbbgfvIDv3cAVSaWSyoMw/YEzxuBuoBgudnAG255Xlec4S73+Xuje7eWFtbe8I7lKvsyMLwGgYSETnWiAHg7l9293nu3kB2EvcJd/8k8CTwkaDaSuDB4PHqYJvg+Sfc3YPym4OzhBYAi4Hnx2xP8kgGl4TWFUFFRI4XG7nKkP4BuM/M/hvwInB3UH438DMzayL7l//NAO7+upn9CngDSAG3ufu4fjO/MwSkABAROdYJBYC7PwU8FTzeSp6zeNy9F7hpiNd/A/jGiTbyZJUpAEREhhTqXwInNQcgIjKkUAfAO8tC6iwgEZFjhTsAEpoEFhEZSqgDQJPAIiJDUwCIiBSpUAfAkSEgzQGIiBwn1AGQiEaIRkw9ABGRPEIdAGZGUstCiojkFeoAgOwwUK/OAhIROU7oA0BrAoiI5Bf6AChLxBQAIiJ5hD4AkokoPQM6C0hE5FhFEQDqAYiIHC/0AVAaj+picCIieYQ+ANQDEBHJTwEgIlKkQh8AZfGYLgUhIpJH6AMgmYjSPZAmuyyxiIgMCn0AlCWiuENfKlPopoiITCqhDwBdElpEJL+iCQCtCiYicrTQB0DZkYXhNREsIpIr9AGQjGsISEQkn/AHgOYARETyCn0AlB5ZFlIBICKSK/QBoB6AiEh+oQ+A6mQCgD0HewrcEhGRySX0AVA3vZQlsyp48s2WQjdFRGRSCX0AAFx51iye39ZGR+9AoZsiIjJpFEUAXHV2HamM88e39he6KSIik0ZRBMDS+koqk3Ge2KRhIBGRQUURALFohPctqeWpN1tIZ3RVUBERGEUAmFmpmT1vZi+b2etm9tWgfIGZPWdmm83sl2aWCMpLgu2m4PmGnPf6clD+ppldO147lc+VZ9VxoKufl5sPTuTHiohMWqPpAfQBV7r7+cAFwHVmthz4FvAdd18MtAO3BPVvAdrdfRHwnaAeZnYOcDPwLuA64AdmFh3LnRnO+5bUEo0YT2zUMJCICIwiADyrM9iMBzcHrgTuD8pXATcGj1cE2wTPX2VmFpTf5+597r4NaAKWjclejEJlMsFFp1fxuOYBRESAUc4BmFnUzF4CWoA1wBbgoLsPXmKzGZgbPJ4L7AIInj8EzMwtz/Oa3M+61czWm9n61tbWE9+jYVx1Vh0b93boR2EiIowyANw97e4XAPPI/tV+dr5qwb0N8dxQ5cd+1l3u3ujujbW1taNp3qhdeVYdgM4GEhHhBM8CcveDwFPAcqDSzGLBU/OAPcHjZqAeIHh+BtCWW57nNRNiUV0Fc2aU8vRm/R5ARGQ0ZwHVmlll8LgM+ACwEXgS+EhQbSXwYPB4dbBN8PwTnl2RfTVwc3CW0AJgMfD8WO3IaJgZly2u4c9b9ut0UBEpeqPpAcwGnjSzV4B1wBp3fwj4B+DvzKyJ7Bj/3UH9u4GZQfnfAbcDuPvrwK+AN4BHgNvcfcIv0Xnpoho6elO8uvvQRH+0iMikEhupgru/AizNU76VPGfxuHsvcNMQ7/UN4Bsn3syxc+miGgCe3tzKBfWVhWyKiEhBFcUvgXPVVJRwzuzpPN2keQARKW5FFwAA711cw4Yd7XRroXgRKWJFGQCXLqphIO08t62t0E0RESmYogyAZQuqScQi/Emng4pIESvKACiNR7m4oUrzACJS1IoyACA7DLRp32FaDvcWuikiIgVRtAHw3kXZy0z8Sb0AESlSRRsA75oznZqKEh5+dV+hmyIiUhBFGwCRiPHhC+fyxKYWDQOJSFEq2gAAuKmxnnTGeeCF3YVuiojIhCvqAFhUV0Hj6VX8at0usterExEpHkUdAAAfvbierfu72LCjvdBNERGZUEUfAB88bzbliSi/XLdr5MoiIiFS9AFQXhLj354/h9+9upfOPl0bSESKR9EHAGSHgbr70/z25QldoExEpKAUAMDS+krOmT2dHzzVRO/AhK9RIyJSEAoAsktF/uMNZ7OrrYef/nl7oZsjIjIhFACByxbXcNVZdfzzE03s7+wrdHNERMadAiDHP37wbHoH0nx7zVuFboqIyLhTAORYWFvBp5afzn3P72TTvo5CN0dEZFwpAI7xhQ8sZlppnO8/uaXQTRERGVcKgGNUJhN84OxZPL25lUxGl4cQkfBSAORx2eKZtHcP8MZeDQOJSHgpAPJ4z8IaQIvFiEi4KQDymDW9lEV1Ffxpy4FCN0VEZNwoAIZw2aIa1m1roz+VKXRTRETGhQJgCO9ZOJOegTQv7tRlokUknBQAQ7jkjJlETPMAIhJeCoAhzCiLc968Ss0DiEhoKQCGcdmimby866DWCRCRUFIADOPShTWkMs7z29QLEJHwGTEAzKzezJ40s41m9rqZfT4orzazNWa2ObivCsrNzL5nZk1m9oqZXZjzXiuD+pvNbOX47dbYuPD0KkpiEZ7erAAQkfAZTQ8gBXzJ3c8GlgO3mdk5wO3A4+6+GHg82Aa4Hlgc3G4F7oRsYABfAS4BlgFfGQyNyao0HuWSM2by6Ov7dFkIEQmdEQPA3fe6+wvB48PARmAusAJYFVRbBdwYPF4B3OtZzwKVZjYbuBZY4+5t7t4OrAGuG9O9GQcfbZzH7oM9rN3cWuimiIiMqROaAzCzBmAp8Bwwy933QjYkgLqg2lxgV87LmoOyocqP/YxbzWy9ma1vbS38l+4155zGzPIEv3h+Z6GbIiIypkYdAGZWAfwa+IK7D3eVNMtT5sOUH13gfpe7N7p7Y21t7WibN24SsQgfaZzHYxtbeLujt9DNEREZM6MKADOLk/3y/7m7PxAUvx0M7RDctwTlzUB9zsvnAXuGKZ/0Pn7xfNIZ51/X7xq5sojIFDGas4AMuBvY6O7fznlqNTB4Js9K4MGc8k8HZwMtBw4FQ0SPAteYWVUw+XtNUDbpNdSUc+mimfzi+V2kNRksIiExmh7ApcBfAVea2UvB7Qbgm8DVZrYZuDrYBngY2Ao0AT8GPgfg7m3A14F1we1rQdmU8PFl89l9sIc/ajJYREIiNlIFd3+a/OP3AFflqe/AbUO81z3APSfSwMlicDL458/t5Ioz60Z+gYjIJKdfAo9SIhbhE5fM57GNb9PUcrjQzREROWUKgBPwmUsXUBqL8gMtGC8iIaAAOAHV5Qk+tXw+D768hx0HugrdHBGRU6IAOEH/4b1nEI0YP/yDegEiMrUpAE5Q3fRSbr64nvs3NLPnYE+hmyMictIUACfhb963EHe4a+3WQjdFROSkKQBOwtzKMv5y6VzuW7eTjt6BQjdHROSkKABO0qff3UDvQIYHX9xd6KaIiJwUBcBJOm/eDM6dO52fP7eT7G/fRESmFgXAKfj4svls2neYl3YdLHRTREROmALgFKy4YC7JRFRrBYjIlKQAOAUVJTFWXDCH3768V5PBIjLlKABO0SeWnU7PQFqTwSIy5SgATlHuZLCIyFSiABgDH2usZ9O+w2zaN9xKmSIik4sCYAzccN5sohHjwZemxAqXIiKAAmBMzKwo4bJFNfz25T36TYCITBkKgDGy4oI5NLf38MJO/SZARKYGBcAYueZdp1ESi7D6JZ0NJCJTgwJgjFSUxLjq7Dp+9+peUulMoZsjIjIiBcAY+tD5c9jf2c8zWw8UuikiIiNSAIyhK86sY1pJTGcDiciUECt0A8KkNB7l2nNP46FX9pBMRLn+3NksW1BNNGKFbpqIyHEUAGPsi1cvoasvxa/W7+LeZ3ZQlYxz0elVLJ1fxSULqrno9CrMFAgiUngKgDE2t7KMOz91Ed39KZ56s5UnNrXwws52HtvYAsAnLpnPVz/0LuJRjb6JSGEpAMZJMhHjhvNmc8N5swFo7+rnh2u38KM/bGVLSyd3fuoiqssTBW6liBQz/Rk6QarKE3z5+rO542MX8OKug6z4/tPs7+wrdLNEpIgpACbYjUvn8rO/Xsauth4efnVvoZsjIkVMAVAAyxZUU19dxtq3WgvdFBEpYgqAAjAzLl9cyzNbDtCf0q+GRaQwFAAFcvmSWrr607yws73QTRGRIjViAJjZPWbWYmav5ZRVm9kaM9sc3FcF5WZm3zOzJjN7xcwuzHnNyqD+ZjNbOT67M3W8Z+FMYhHTMJCIFMxoegA/Ba47pux24HF3Xww8HmwDXA8sDm63AndCNjCArwCXAMuArwyGRrGaVhrnwvlVrN2sABCRwhgxANx9LdB2TPEKYFXweBVwY075vZ71LFBpZrOBa4E17t7m7u3AGo4PlaJz+ZIaXtvdodNBRaQgTnYOYJa77wUI7uuC8rnArpx6zUHZUOVF7fIltQA8vXl/gVsiIsVorCeB813kxocpP/4NzG41s/Vmtr61NdzDI+fOmUF1eULzACJSECcbAG8HQzsE9y1BeTNQn1NvHrBnmPLjuPtd7t7o7o21tbUn2bypIRIxLltUw9rN+8lktJawiEyskw2A1cDgmTwrgQdzyj8dnA20HDgUDBE9ClxjZlXB5O81QVnRu3xJLfs7+3hxl9YSFpGJNZrTQH8BPAOcaWbNZnYL8E3gajPbDFwdbAM8DGwFmoAfA58DcPc24OvAuuD2taCs6F15Vh3V5QluWbWOZ7WSmIhMIHOfvEMPjY2Nvn79+kI3Y9ztONDFX/90HTvbuvnv/+7f8JGL5hW6SSIyhZnZBndvHLGeAmByONQ9wH/8+Qb+vOUA582dwXsX13D5kloubtCKYiJyYhQAU9BAOsM9T2/jsY1v88LOg6QzztL5lfyvm87njNqKQjdPRKYIBcAU19E7wCOv7uMbD2+kL5XmH647i5XvbiCi3oCIjGC0AaCLwU1S00vjfPTien7/xctZfsZMvvrbN/jET55l54HuQjdNREJCATDJzZpeyr/8+4v51ofP4/XdHVz33bXc+8x2/W5ARE6ZhoCmkD0He7j9gVdZ+1Yr00pj1FclmV+d5KbGeVx19qxCN09EJonRDgFpUfgpZE5lGas+czEPvbKXddvb2NXWzcvNB3liUwsPfO49nDt3RqGbKCJTiHoAU1xbVz83fPePlMQjPPSfL2NaabzQTRKRAtMkcJGoLk/wvz+xlOb27PDQZA50EZlcFAAhcHFDNV+6Zgm/e2Uv3/x/m3hjTwdpTRKLyAg0BxASn718Ia/sOsSP1m7lR2u3Mq0kxgXzK1laX8n59ZUsmTWNipIYFaUx4lHlvogoAEIjEjHu/NSF7D7Yw7rtbTy/rZ0Xd7bzz082cWxnIB41EtEIiViEZCLG/OokDTXlNMxMUje9hJqKEk6bXsoZtRW6DIVIiCkAQsTMmFeVZF5Vkr9cmr2gXHd/itf3dLBtfxddfSk6e1N09acZSGfoT2U43DvAjrZuHnltL+3dA0e9X0VJjAtPr+LC+ZXUVJQwrTTG9NI4y8+YSVkiWohdFJExpAAIuWQixsUN1VzcUD1i3cO9A7Qe7mN/Zz/N7d1s2NHO+u3t3PHY5qPq1VSU8DeXn8Enl88nmdB/IZGpSqeByoh6+tN09A7Q2ZdiV1s3P/7jVv7UdICaigRXnFnHuXOmc+7cGZxfX6n5BZFJQD8EkzFTlohSlogyC1hYW8EVZ9axfnsbd63dypObWrh/QzMANRUJPnzRPD7WWK+rl4pMAeoByClxd97u6OPFne385sXdPL6phXTG+dD5c/j6jecyo0w/TBOZaOoByIQwM06bUcr1583m+vNm09LRy73P7ODOP2xhw4527rj5glHNP4jIxNOArYypuuml/Jdrz+T+z76baMT42I+e4Wu/fYOWjt5CN01EjqEAkHGxdH4Vv/vby/hoYz0//fM2LvsfT/JfH3yN1/ccoi+VLnTzRATNAcgE2L6/ix/+YQu/fqGZgbQTjRgLasp59xkz+dz7FzJ7RlmhmygSKloSUiadfYd6eX57G2/tO8ymfYf5w1stmBl/tfx0Pvu+hdROKyl0E0VCQQEgk15zezfffWwzv36hmYzDoroKltZXcu7cGdRNK6FmWgkzyxNUlMaoKIlRFo9ipktTiIxEASBTxpbWTh55bR8v7GjnhZ3tx12SYlA0YlQl41SXJ6guTzBnRhlzKss4bUYp00pjlMSyv1cYvM5RSSxCMhGloiRGeUmM0nhU1zaSoqDTQGXKWFhbwW3vXwRkf1fQ2tnH/sP9tHb20dbVR2dfmq6+FB09A7R3D9De1c/+zj6e29bGvo7eE7r0dTRiORfDi1ISy4ZFPGokYhFikQjRiBG17HZpPBsqM8pizKksY25lGfXVSRbXVWjxHZnyFAAyqZgZddNKqZtWOqr6qXSG/Z39dPen6BlI0zuQpi+VvdBdXypDT3+aw30puvpS9A1k6E+n6U9lGEj7kXr96QwDg/fpDBl30hmnuz/Fga5+egfStHX1c6jn6J7JnBmlLKgtp7IswfSyONXlcRbUVLCoroKGmUmmlcbV45BJTQEgU1osGuG0GaMLi1PV1Zdi76Eetu/v5q2Ww7y17zA72rrZd6iDQz0p2rv7j+uNlCeiJEtiDOaAYcwoi1NVHqcqmaAsHiUeDFlFI3bklu2lRIPyd96vJBZlRlmcGWVxppVm13coT8RIJqLEYxES0ezQl+ZKZDQUACKjVF4SY1HdNBbVTeMD58w67vmBdIadbd1saelkZ1s3h3tTdPal6O5PMTjVls44Hb0DtHcN0NTSSW8q2yPpT2VIZ7I9j7Q7A2k/6VXdBudKqpIJppXGjgqDqBmRCJTGo8yeUcqcGWXMml56pKdiBslElPJg3iQ6RJAkYhHKguGx0ng0CDJT8EwxCgCRMRKPRlhYW8HCMboQXjrj9KeyQ1IADvQOpOnoGeBQzwCHe7NDW4f7UvQOpI8Me3X3p47MlRzuTR15PycbKpkMHOjs57XdHezv7BuTtkI2eGorSmioSbKgppyaihIS0QjxWIRYzlBYRUmMxbOmsWSW5lEKTQEgMklFI3bcwjsVJTFqKsbu9xK9A2laD/cd6aFk3OnuT9PVn+295DtL0D3b2+kZSNPdn6anP30kePYe6mX7/i4eff1t2rv7Gekkw6pknJJY9MhEfCwSIRIxElFjelmc6aXZoa5Y1ILeixGLvHNfGpz5VRKPDtlbiRjvvC6nzuDnDE7+D77vkaE4y95HzIhFLRjGy/8Z0Zx2Db4uFjFK49kTDSKTdC5IASBSxErjUeqrk+Py3h5MpvenM6RyhrMOdg3w1tuHefPtw+w52HNkdbrBYa+0Z3s+Hb0D7D7YQ2dvinTGSWWcTPB8KuOk0pnjljudrEpiEeLRCBHjSKgMikayIZSIRY4KsSvOrOWfPnjOuLZrwgPAzK4DvgtEgZ+4+zcnug0iMv4s+Ms5dswiQdNL48yfmcw7j3KiBnsivQPpIXsbGXdSQbjkVhkcYutPZ0ilj56DOfI448FZYZAe4gPcc+pkMkfuUxkPekbZ9qXSfuQMs8GWuL/Tjr505qge16zp439yw4QGgJlFge8DVwPNwDozW+3ub0xkO0QkHOLR7F/W0zWXcFIm+mqgy4Amd9/q7v3AfcCKCW6DiIgw8QEwF9iVs90clImIyASb6ADINxV+1MCamd1qZuvNbH1ra+sENUtEpPhMdAA0A/U52/OAPbkV3P0ud29098ba2toJbZyISDGZ6ABYByw2swVmlgBuBlZPcBtERIQJPgvI3VNm9p+AR8meBnqPu78+kW0QEZGsCf8dgLs/DDw80Z8rIiJH06LwIiJFalKvCGZmrcCOU3iLGmD/GDVnqijGfYbi3G/tc/E40f0+3d1HPItmUgfAqTKz9aNZFi1MinGfoTj3W/tcPMZrvzUEJCJSpBQAIiJFKuwBcFehG1AAxbjPUJz7rX0uHuOy36GeAxARkaGFvQcgIiJDCGUAmNl1ZvammTWZ2e2Fbs94MLN6M3vSzDaa2etm9vmgvNrM1pjZ5uC+qtBtHQ9mFjWzF83soWB7gZk9F+z3L4NLjYSGmVWa2f1mtik45u8uhmNtZl8M/n+/Zma/MLPSMB5rM7vHzFrM7LWcsrzH17K+F3y/vWJmF57s54YuAHIWnbkeOAf4uJmN77pqhZECvuTuZwPLgduC/bwdeNzdFwOPB9th9HlgY872t4DvBPvdDtxSkFaNn+8Cj7j7WcD5ZPc91MfazOYCfws0uvu5ZC8fczPhPNY/Ba47pmyo43s9sDi43QrcebIfGroAoEgWnXH3ve7+QvD4MNkvhLlk93VVUG0VcGNhWjh+zGwe8EHgJ8G2AVcC9wdVQrXfZjYduBy4G8Dd+939IEVwrMlerqbMzGJAEthLCI+1u68F2o4pHur4rgDu9axngUozm30ynxvGACi6RWfMrAFYCjwHzHL3vZANCaCucC0bN3cAfw9kgu2ZwEF3TwXbYTvmZwCtwL8Ew14/MbNyQn6s3X038D+BnWS/+A8BGwj3sc411PEds++4MAbAiIvOhImZVQC/Br7g7h2Fbs94M7O/AFrcfUNucZ6qYTrmMeBC4E53Xwp0EbLhnnyCMe8VwAJgDlBOdvjjWGE61qMxZv/fwxgAIy46ExZmFif75f9zd38gKH57sDsY3LcUqn3j5FLgQ2a2nezw3pVkewSVwTABhO+YNwPN7v5csH0/2UAI+7H+ALDN3VvdfQB4AHgP4T7WuYY6vmP2HRfGACiKRWeCce+7gY3u/u2cp1YDK4PHK4EHJ7pt48ndv+zu89y9geyxfcLdPwk8CXwkqBaq/Xb3fcAuMzszKLoKeIOQH2uyQz/LzSwZ/H8f3O/QHutjDHV8VwOfDs4GWg4cGhwqOmHuHrobcAPwFrAF+KdCt2ec9vEyst2+V4CXgtsNZMfDHwc2B/fVhW7rOP4bXAE8FDw+A3geaAL+FSgpdPvGeF8vANYHx/v/AlXFcKyBrwKbgNeAnwElYTzWwC/IznMMkP0L/5ahji/ZIaDvB99vr5I9S+qkPle/BBYRKVJhHAISEZFRUACIiBQpBYCISJFSAIiIFCkFgIhIkVIAiIgUKQWAiEiRUgCIiBSp/w8S+gpDZDh+hAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x,frequenices)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 使用2—gram语言模型验证语言的合理性"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### 数学公式抽象化：计算相邻单词的概率"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 222,
   "metadata": {},
   "outputs": [],
   "source": [
    "def prob_1(word):\n",
    "    return words_count[word] / len(TOKEN)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 223,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.026804365991226237"
      ]
     },
     "execution_count": 223,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prob_1('我')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 224,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['法律', '要求', '残疾', '保险', '吗', '债权人', '可以', '在', '死', '后']"
      ]
     },
     "execution_count": 224,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "TOKEN[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 225,
   "metadata": {},
   "outputs": [],
   "source": [
    "TOKEN = [str(s) for s in TOKEN]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 226,
   "metadata": {},
   "outputs": [],
   "source": [
    "TOKEN_2_GRAM = [''.join(TOKEN[i:i+2]) for i in range(len(TOKEN[:-2]))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 227,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['法律要求', '要求残疾', '残疾保险', '保险吗', '吗债权人', '债权人可以', '可以在', '在死', '死后', '后人寿保险']"
      ]
     },
     "execution_count": 227,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "TOKEN_2_GRAM[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 228,
   "metadata": {},
   "outputs": [],
   "source": [
    "words_count_2 = Counter(TOKEN_2_GRAM)  #Counter函数返回字典对象"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 229,
   "metadata": {},
   "outputs": [],
   "source": [
    "def prob_2(word1,word2):\n",
    "    if word1+word2 in TOKEN_2_GRAM: return words_count_2[word1+word2]/len(TOKEN_2_GRAM)\n",
    "    else:\n",
    "        return 1/len(TOKEN_2_GRAM)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 230,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.3056534795665231e-05"
      ]
     },
     "execution_count": 230,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prob_2('我们','在')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 231,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.3056534795665231e-05"
      ]
     },
     "execution_count": 231,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prob_2('在','吃饭')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### 计算整个句子存在的合理性概率"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 232,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cut(string): return list(jieba.cut(string))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 233,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_probablity(sentence):\n",
    "    words = cut(sentence)\n",
    "    sentence_prob = 1\n",
    "    \n",
    "    for i,word in enumerate(words[:-1]):\n",
    "        next_ = words[i+1]\n",
    "        words_2_prob = prob_2(word,next_)\n",
    "        sentence_prob *= words_2_prob\n",
    "        \n",
    "    return sentence_prob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 234,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.954132101817813e-30"
      ]
     },
     "execution_count": 234,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_probablity('小明今天抽奖抽到一台苹果手机')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 240,
   "metadata": {},
   "outputs": [],
   "source": [
    "need_repared = [generate(create_grammar(first_grammar),'ninja') for _ in range(30)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 241,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['千鸟打败了一个村的千鸟',\n",
       " '一个小队的忍术强的佐助使用一个村的火之国的火之国的螺旋丸',\n",
       " '一个无限月读击溃了一个忍术强的水之国的体术强的螺旋丸',\n",
       " '须佐能乎使用水之国的宇智波班',\n",
       " '这个忍术强的鸣人击溃了一个小队的神罗天征',\n",
       " '一个须佐能乎使用一个小队的佩恩',\n",
       " '一个小队的千鸟击败了无限月读',\n",
       " '一个村的体术强的佐助打倒了这个螺旋手里剑',\n",
       " '这个体术强的幻术强的幻术强的体术强的须佐能乎击溃了一个小队的忍术强的体术强的佩恩',\n",
       " '这个佩恩击败了一个佐助',\n",
       " '这个体术强的须佐能乎击败了一个村的火之国的火之国的无限月读',\n",
       " '一个千鸟打倒了这个辉夜',\n",
       " '这个火之国的火之国的水之国的螺旋丸击败了这个辉夜',\n",
       " '这个幻术强的辉夜击溃了这个火之国的鸣人',\n",
       " '一个宇智波班使用一个须佐能乎',\n",
       " '佩恩使用一个螺旋手里剑',\n",
       " '这个忍术强的无限月读击败了一个村的宇智波班',\n",
       " '体术强的宇智波班击溃了忍术强的火之国的体术强的鸣人',\n",
       " '这个火之国的宇智波班打倒了一个火之国的千鸟',\n",
       " '这个螺旋丸打倒了一个村的火之国的火之国的佐助',\n",
       " '一个小队的佩恩击溃了这个螺旋手里剑',\n",
       " '一个体术强的水之国的幻术强的辉夜使用一个螺旋丸',\n",
       " '这个幻术强的水之国的水之国的水之国的佩恩击败了一个村的幻术强的鸣人',\n",
       " '一个忍术强的鸣人击败了一个螺旋丸',\n",
       " '一个小队的神罗天征击溃了一个村的火之国的忍术强的鸣人',\n",
       " '一个小队的须佐能乎打败了一个小队的火之国的体术强的鸣人',\n",
       " '一个小队的忍术强的螺旋手里剑打倒了一个忍术强的水之国的佩恩',\n",
       " '一个村的佐助击溃了这个千鸟',\n",
       " '须佐能乎打败了一个村的水之国的忍术强的宇智波班',\n",
       " '一个小队的水之国的鸣人击溃了这个鸣人']"
      ]
     },
     "execution_count": 241,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "need_repared"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 253,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compare_better(need_repared):\n",
    "    '''比较自动生成相邻句子哪个存在更合理'''\n",
    "    for i,s1 in enumerate(need_repared[:-1]):\n",
    "        s2 = need_repared[i+1]\n",
    "        p1,p2 = get_probablity(s1),get_probablity(s2)\n",
    "        \n",
    "        better = s1 if p1 > p2 else s2\n",
    "        print('{} is more possible'.format(better))\n",
    "        print('-'*4 + ' {} with probility {}'.format(s1, p1))\n",
    "        print('-'*4 + ' {} with probility {}'.format(s2, p2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 254,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "千鸟打败了一个村的千鸟 is more possible\n",
      "---- 千鸟打败了一个村的千鸟 with probility 3.4678924712724686e-29\n",
      "---- 一个小队的忍术强的佐助使用一个村的火之国的火之国的螺旋丸 with probility 7.132583878449733e-79\n",
      "一个无限月读击溃了一个忍术强的水之国的体术强的螺旋丸 is more possible\n",
      "---- 一个小队的忍术强的佐助使用一个村的火之国的火之国的螺旋丸 with probility 7.132583878449733e-79\n",
      "---- 一个无限月读击溃了一个忍术强的水之国的体术强的螺旋丸 with probility 4.992808714914813e-78\n",
      "须佐能乎使用水之国的宇智波班 is more possible\n",
      "---- 一个无限月读击溃了一个忍术强的水之国的体术强的螺旋丸 with probility 4.992808714914813e-78\n",
      "---- 须佐能乎使用水之国的宇智波班 with probility 2.9061078120375347e-20\n",
      "须佐能乎使用水之国的宇智波班 is more possible\n",
      "---- 须佐能乎使用水之国的宇智波班 with probility 2.9061078120375347e-20\n",
      "---- 这个忍术强的鸣人击溃了一个小队的神罗天征 with probility 1.3158466382127003e-53\n",
      "一个须佐能乎使用一个小队的佩恩 is more possible\n",
      "---- 这个忍术强的鸣人击溃了一个小队的神罗天征 with probility 1.3158466382127003e-53\n",
      "---- 一个须佐能乎使用一个小队的佩恩 with probility 4.954132101817813e-30\n",
      "一个须佐能乎使用一个小队的佩恩 is more possible\n",
      "---- 一个须佐能乎使用一个小队的佩恩 with probility 4.954132101817813e-30\n",
      "---- 一个小队的千鸟击败了无限月读 with probility 8.445462615185585e-40\n",
      "一个小队的千鸟击败了无限月读 is more possible\n",
      "---- 一个小队的千鸟击败了无限月读 with probility 8.445462615185585e-40\n",
      "---- 一个村的体术强的佐助打倒了这个螺旋手里剑 with probility 1.8797809117324288e-54\n",
      "一个村的体术强的佐助打倒了这个螺旋手里剑 is more possible\n",
      "---- 一个村的体术强的佐助打倒了这个螺旋手里剑 with probility 1.8797809117324288e-54\n",
      "---- 这个体术强的幻术强的幻术强的体术强的须佐能乎击溃了一个小队的忍术强的体术强的佩恩 with probility 9.385386518228187e-132\n",
      "这个佩恩击败了一个佐助 is more possible\n",
      "---- 这个体术强的幻术强的幻术强的体术强的须佐能乎击溃了一个小队的忍术强的体术强的佩恩 with probility 9.385386518228187e-132\n",
      "---- 这个佩恩击败了一个佐助 with probility 2.6560588437475834e-24\n",
      "这个佩恩击败了一个佐助 is more possible\n",
      "---- 这个佩恩击败了一个佐助 with probility 2.6560588437475834e-24\n",
      "---- 这个体术强的须佐能乎击败了一个村的火之国的火之国的无限月读 with probility 8.511395836843695e-88\n",
      "一个千鸟打倒了这个辉夜 is more possible\n",
      "---- 这个体术强的须佐能乎击败了一个村的火之国的火之国的无限月读 with probility 8.511395836843695e-88\n",
      "---- 一个千鸟打倒了这个辉夜 with probility 3.7943697767822626e-25\n",
      "一个千鸟打倒了这个辉夜 is more possible\n",
      "---- 一个千鸟打倒了这个辉夜 with probility 3.7943697767822626e-25\n",
      "---- 这个火之国的火之国的水之国的螺旋丸击败了这个辉夜 with probility 2.454342488226177e-59\n",
      "这个幻术强的辉夜击溃了这个火之国的鸣人 is more possible\n",
      "---- 这个火之国的火之国的水之国的螺旋丸击败了这个辉夜 with probility 2.454342488226177e-59\n",
      "---- 这个幻术强的辉夜击溃了这个火之国的鸣人 with probility 1.8797809117324288e-54\n",
      "一个宇智波班使用一个须佐能乎 is more possible\n",
      "---- 这个幻术强的辉夜击溃了这个火之国的鸣人 with probility 1.8797809117324288e-54\n",
      "---- 一个宇智波班使用一个须佐能乎 with probility 2.9061078120375347e-20\n",
      "佩恩使用一个螺旋手里剑 is more possible\n",
      "---- 一个宇智波班使用一个须佐能乎 with probility 2.9061078120375347e-20\n",
      "---- 佩恩使用一个螺旋手里剑 with probility 2.9061078120375347e-20\n",
      "佩恩使用一个螺旋手里剑 is more possible\n",
      "---- 佩恩使用一个螺旋手里剑 with probility 2.9061078120375347e-20\n",
      "---- 这个忍术强的无限月读击败了一个村的宇智波班 with probility 1.718039741758324e-58\n",
      "这个忍术强的无限月读击败了一个村的宇智波班 is more possible\n",
      "---- 这个忍术强的无限月读击败了一个村的宇智波班 with probility 1.718039741758324e-58\n",
      "---- 体术强的宇智波班击溃了忍术强的火之国的体术强的鸣人 with probility 5.46284599250465e-74\n",
      "这个火之国的宇智波班打倒了一个火之国的千鸟 is more possible\n",
      "---- 体术强的宇智波班击溃了忍术强的火之国的体术强的鸣人 with probility 5.46284599250465e-74\n",
      "---- 这个火之国的宇智波班打倒了一个火之国的千鸟 with probility 7.718793355046233e-44\n",
      "这个火之国的宇智波班打倒了一个火之国的千鸟 is more possible\n",
      "---- 这个火之国的宇智波班打倒了一个火之国的千鸟 with probility 7.718793355046233e-44\n",
      "---- 这个螺旋丸打倒了一个村的火之国的火之国的佐助 with probility 1.718039741758324e-58\n",
      "一个小队的佩恩击溃了这个螺旋手里剑 is more possible\n",
      "---- 这个螺旋丸打倒了一个村的火之国的火之国的佐助 with probility 1.718039741758324e-58\n",
      "---- 一个小队的佩恩击溃了这个螺旋手里剑 with probility 8.445462615185585e-40\n",
      "一个小队的佩恩击溃了这个螺旋手里剑 is more possible\n",
      "---- 一个小队的佩恩击溃了这个螺旋手里剑 with probility 8.445462615185585e-40\n",
      "---- 一个体术强的水之国的幻术强的辉夜使用一个螺旋丸 with probility 3.204520809800466e-64\n",
      "一个体术强的水之国的幻术强的辉夜使用一个螺旋丸 is more possible\n",
      "---- 一个体术强的水之国的幻术强的辉夜使用一个螺旋丸 with probility 3.204520809800466e-64\n",
      "---- 这个幻术强的水之国的水之国的水之国的佩恩击败了一个村的幻术强的鸣人 with probility 1.4509640410423017e-97\n",
      "一个忍术强的鸣人击败了一个螺旋丸 is more possible\n",
      "---- 这个幻术强的水之国的水之国的水之国的佩恩击败了一个村的幻术强的鸣人 with probility 1.4509640410423017e-97\n",
      "---- 一个忍术强的鸣人击败了一个螺旋丸 with probility 1.0078069402071071e-48\n",
      "一个忍术强的鸣人击败了一个螺旋丸 is more possible\n",
      "---- 一个忍术强的鸣人击败了一个螺旋丸 with probility 1.0078069402071071e-48\n",
      "---- 一个小队的神罗天征击溃了一个村的火之国的忍术强的鸣人 with probility 3.823992194753255e-73\n",
      "一个小队的神罗天征击溃了一个村的火之国的忍术强的鸣人 is more possible\n",
      "---- 一个小队的神罗天征击溃了一个村的火之国的忍术强的鸣人 with probility 3.823992194753255e-73\n",
      "---- 一个小队的须佐能乎打败了一个小队的火之国的体术强的鸣人 with probility 6.518878071438586e-83\n",
      "一个小队的忍术强的螺旋手里剑打倒了一个忍术强的水之国的佩恩 is more possible\n",
      "---- 一个小队的须佐能乎打败了一个小队的火之国的体术强的鸣人 with probility 6.518878071438586e-83\n",
      "---- 一个小队的忍术强的螺旋手里剑打倒了一个忍术强的水之国的佩恩 with probility 4.992808714914812e-78\n",
      "一个村的佐助击溃了这个千鸟 is more possible\n",
      "---- 一个小队的忍术强的螺旋手里剑打倒了一个忍术强的水之国的佩恩 with probility 4.992808714914812e-78\n",
      "---- 一个村的佐助击溃了这个千鸟 with probility 6.46837981697064e-35\n",
      "一个村的佐助击溃了这个千鸟 is more possible\n",
      "---- 一个村的佐助击溃了这个千鸟 with probility 6.46837981697064e-35\n",
      "---- 须佐能乎打败了一个村的水之国的忍术强的宇智波班 with probility 1.3158466382127003e-53\n",
      "一个小队的水之国的鸣人击溃了这个鸣人 is more possible\n",
      "---- 须佐能乎打败了一个村的水之国的忍术强的宇智波班 with probility 1.3158466382127003e-53\n",
      "---- 一个小队的水之国的鸣人击溃了这个鸣人 with probility 1.4397242002958673e-49\n"
     ]
    }
   ],
   "source": [
    "compare_better(need_repared)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 257,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "明天晚上请你吃大餐，我们一起吃苹果 is more possible\n",
      "---- 今天晚上请你吃大餐，我们一起吃日料 with probility 1.4397242002958673e-49\n",
      "---- 明天晚上请你吃大餐，我们一起吃苹果 with probility 1.4397242002958673e-49\n",
      "真是一只好看的小猫 is more possible\n",
      "---- 真事一只好看的小猫 with probility 3.7943697767822626e-25\n",
      "---- 真是一只好看的小猫 with probility 2.9061078120375347e-20\n",
      "今晚我去吃火锅 is more possible\n",
      "---- 今晚我去吃火锅 with probility 2.2257879732395474e-15\n",
      "---- 今晚火锅去吃我 with probility 2.9061078120375347e-20\n",
      "养乐多绿来一杯 is more possible\n",
      "---- 洋葱奶昔来一杯 with probility 2.2257879732395474e-15\n",
      "---- 养乐多绿来一杯 with probility 1.7047310087041692e-10\n"
     ]
    }
   ],
   "source": [
    "need_compared_test = [\n",
    "    \"今天晚上请你吃大餐，我们一起吃日料 明天晚上请你吃大餐，我们一起吃苹果\",\n",
    "    \"真事一只好看的小猫 真是一只好看的小猫\",\n",
    "    \"今晚我去吃火锅 今晚火锅去吃我\",\n",
    "    \"洋葱奶昔来一杯 养乐多绿来一杯\"\n",
    "]\n",
    "for s in need_compared_test:\n",
    "    s1,s2 = s.split()\n",
    "    p1,p2 = get_probablity(s1),get_probablity(s2)\n",
    "    \n",
    "    better = s1 if p1 > p2 else s2\n",
    "    print('{} is more possible'.format(better))\n",
    "    print('-'*4 + ' {} with probility {}'.format(s1, p1))\n",
    "    print('-'*4 + ' {} with probility {}'.format(s2, p2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.获得最优质的语言"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 259,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_best(grammar,target,n):\n",
    "    sentence_prob_dict = {}\n",
    "    \n",
    "    for sentence in [generate(grammar,target) for _ in range(n)]:\n",
    "            sentence_prob = get_probablity(sentence)\n",
    "            sentence_prob_dict[sentence] = sentence_prob\n",
    "    \n",
    "    return sorted(sentence_prob_dict,key = lambda x:x[1],reverse = True)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 269,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'路飞拥有武装色霸气'"
      ]
     },
     "execution_count": 269,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generate_best(create_grammar(second_grammar),'poacher',1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 271,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'路飞拥有见闻色霸气'"
      ]
     },
     "execution_count": 271,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generate_best(create_grammar(second_grammar),'poacher',100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 273,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'千鸟打倒了一个村的火之国的鸣人'"
      ]
     },
     "execution_count": 273,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generate_best(create_grammar(first_grammar),'ninja',500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 283,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "路飞吃过面子果实\n",
      "路飞拥有见闻色霸气\n",
      "路飞吃过武装色霸气\n",
      "路飞吃过霸王色霸气\n",
      "路飞拥有武装色霸气\n",
      "路飞吃过武装色霸气\n",
      "路飞拥有见闻色霸气\n",
      "路飞拥有橡胶果实\n",
      "路飞拥有霸王色霸气\n",
      "路飞拥有面子果实\n"
     ]
    }
   ],
   "source": [
    "for _ in range(10):\n",
    "    print(generate_best(create_grammar(second_grammar),'poacher',1000))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 282,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "千鸟使用这个忍术强的火之国的火之国的体术强的幻术强的火之国的体术强的螺旋丸\n",
      "千鸟打败了一个螺旋丸\n",
      "千鸟打倒了火之国的螺旋手里剑\n",
      "千鸟击溃了一个小队的佩恩\n",
      "无限月读击溃了体术强的须佐能乎\n",
      "千鸟打败了这个辉夜\n",
      "千鸟击败了佩恩\n",
      "千鸟打倒了一个千鸟\n",
      "千鸟击败了一个村的佩恩\n",
      "千鸟打败了一个须佐能乎\n"
     ]
    }
   ],
   "source": [
    "for _ in range(10):\n",
    "    print(generate_best(create_grammar(first_grammar),'ninja',500))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Q: 这个模型有什么问题？ 你准备如何提升？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Ans:问题：挑选的数据集太小，不能正确的验证合理性；数据集是保险行业相关的问答与自己设定的动漫人物的语法对象不相关；"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 提升：使用不同性质不同方面的数据集、掌握爬虫后，爬取大量的相关数据集供测试；了解其他的概率模型并分析出利弊，挑选适合实验数据集的模型"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
